Web: add an opt-in storageBucket option for OPFS databases - #1142

Uses the getRoot and prefix options of the VFS implementations, from rhashimoto/wa-sqlite#380, released in @journeyapps/wa-sqlite 2.0.9. This PR moves the catalog entry to 2.0.10, since 2.0.9 had a build regression.

Summary

• Adds an optional storageBucket open option to @powersync/web. When it is set, the database files of the OPFS-based VFS implementations live in that Storage Bucket instead of the default bucket of the origin private file system.
• Opt-in. When it is not set, which is the default, nothing changes: the same storage, the same behaviour. Existing apps are not affected.
• The SDK holds the Storage Buckets logic, as agreed in the wa-sqlite review: the database worker opens the bucket for the VFS's root directory and prefixes the VFS lock and channel names with the bucket name. The VFS library only takes a root directory function and a prefix.
• Why: a Chromium defect in the default bucket can make every OPFS call of a site fail after the user deletes the site's data, so the app cannot open its database in any tab, and it can silently delete a database that a newer tab has stored. A named bucket is not affected.

const db = new PowerSyncDatabase({
  schema,
  database: {
    dbFilename: 'app.db',
    vfs: WASQLiteVFS.OPFSWriteAheadVFS,
    // Opening fails without the Storage Buckets API, so check for it.
    storageBucket: 'storageBuckets' in navigator ? 'app-db' : undefined
  }
});

The problem

When a site's data is deleted, Chrome replaces the site's default storage bucket. Ordinary ways of deleting it all do this: Chrome's Delete browsing data dialog with its default choices, Site settings → the site → Delete data, and a server response with Clear-Site-Data: "storage", which some apps send on sign-out. The DevTools Clear site data button does not. File and directory handles obtained before the deletion keep referring to the old bucket, while new handles refer to the new one. For the default bucket both refer to the same folder on disk, and Chrome then keeps two indexes for that folder.

• If a handle from before the deletion is used first, every later OPFS call of the site fails with InvalidStateError: An operation that depends on state cached in an interface object was made but the state had changed since it was read from disk. Each call takes about 3 seconds to fail. The database cannot be opened in any tab, reloads do not help, and it lasts until the browser restarts or 10 minutes pass without OPFS use by any site. Retries keep it going.
• If a new tab has already stored its database, a call through an old handle fails and also deletes that database. The new tab gets no error.

A PowerSync tab with an OPFS database open holds such handles, and the VFS uses them when the database is closed, when the tab reloads and when it closes. So an app cannot avoid the defect from outside the SDK. The Chromium internals, and the measurement showing that named buckets are deleted with the site's data like the default bucket, are in the wa-sqlite PR.

Steps to reproduce

With a PowerSync app

1. Open a PowerSync web app that uses WASQLiteVFS.OPFSWriteAheadVFS or OPFSCoopSyncVFS in tab A, and let it open its database.
2. Open chrome://settings/content/siteDetails?site=<the app's origin, URL-encoded>, choose Delete data and confirm with Delete.
3. Open the app in tab B. It creates a new database.
4. Reload tab A.

Result: tab B's database files are deleted. If instead, after step 2, another page only requested the directory before tab A reloaded, every OPFS call of the site fails from then on, and the app cannot open its database in any tab. We see this error in production when our PowerSync app opens its database. The two cases below were measured with OPFSWriteAheadVFS opened the way PowerSync opens it, in a dedicated worker, on Chrome 154.0.8037.98:

In the browser console, without PowerSync

Use a fresh profile and a page on an origin with OPFS, for example http://localhost:8080/. The site needs some stored data, or Chrome does not offer Delete data.

1. Tab A:

   localStorage.setItem('example', '1');
   await (await caches.open('example')).put('/example', new Response(new Uint8Array(5 * 1024 * 1024)));
   window.old = await navigator.storage.getDirectory();

2. Delete the site's data as in step 2 above.

3. Tab A, without reloading: await window.old.getFileHandle('after-delete', { create: true });

4. Tab B:

   const root = await navigator.storage.getDirectory();
   for await (const name of root.keys()) console.log(name);

   This fails with the InvalidStateError after about 3 seconds, and so does every later OPFS call of the site.

The same happens when step 2 uses Chrome's Delete browsing data dialog with its default choices, or a fetch of a same-origin URL whose response carries Clear-Site-Data: "storage". Both were measured on Chrome 154.0.8037.98, with the same results as Site settings.

With (await navigator.storageBuckets.open('app-db')).getDirectory() in place of navigator.storage.getDirectory(), the call in step 3 fails at once and step 4 works. The wa-sqlite PR has a second variant that shows the deleted files, and a table of the measured results.

What this PR changes

• packages/web/src/db/adapters/options.ts: the documented storageBucket option. The documentation covers the feature check, that the name is passed to the browser unchanged, how to configure the bucket (open it with navigator.storageBuckets.open() before opening the database), and that database names should be unique even across buckets.
• resolveAndValidateOptions.ts: setting the option with a VFS that does not keep its files in OPFS is a configuration error.
• WASQLiteOpenFactory.ts and RawSqliteConnection.ts: the option travels to the database worker with the other open options. It is a string, so it crosses the worker boundary; a root directory function could not.
• vfs.ts: vfsOptionsForStorageBucket() maps the bucket name to the VFS options. getRoot opens the bucket each time the VFS asks for its root, so a bucket the browser deleted with the rest of the site's data is created again. prefix is <bucket>:; bucket names cannot contain :, so two buckets never overlap. In a browser without navigator.storageBuckets it throws an error that names the option and the missing API. The three OPFS VFS implementations receive these options.
• pnpm-workspace.yaml and the lockfile: @journeyapps/wa-sqlite 2.0.10.
• A changeset: @powersync/web minor.

Tests

packages/web/tests/storage_bucket.test.ts:

• For each of the three VFS implementations: the database files are in the bucket and not in the default bucket, and a database of the same name in the default bucket keeps its own data.
• vfsOptionsForStorageBucket(): no options without a bucket name; the bucket is opened on each getRoot() call and prefix is the bucket name with :; a missing Storage Buckets API throws.
• A VFS that does not keep its files in OPFS is rejected when the database is constructed.

Run locally with @journeyapps/wa-sqlite 2.0.10 from npm, in Playwright's headless Chromium, as in CI:

• vitest run tests/storage_bucket.test.ts: 10 of 10 pass.
• The whole @powersync/web suite: 25 files, 179 of 179 pass, in two runs. In earlier runs, failures in other files this PR does not touch (main.test.ts, multiple_tabs_iframe.test.ts, triggers.test.ts, multiple_instances.test.ts) came and went between runs.
• The tests read through the writer connection. A reader connection of OPFSWriteAheadVFS can briefly miss a write that just finished, with or without a bucket: in four full-suite runs, a read straight after a write came back empty 6 times in the default storage and once in a bucket.

Compatibility

• Opt-in, per database. Without the option nothing changes.
• A database in a bucket is a different database from one of the same name in the default bucket. Existing files are not moved: the first open with a bucket starts with an empty database, which then syncs. Pending uploads in the default bucket's database are not carried over, so an app that switches should do it when its upload queue is empty.
• The bucket is opened with the browser's default bucket options. To configure it, for example for persistence, the app calls and awaits navigator.storageBuckets.open() with those options before opening the PowerSync database.
• The option needs the Storage Buckets API, so apps should check for navigator.storageBuckets before setting it. The API is in Chromium-based browsers since version 122, and not in Firefox or Safari (caniuse).